vfs: integrate with CJS and ESM module loaders by mcollina · Pull Request #63653 · nodejs/node

mcollina · 2026-05-30T05:41:48Z

Integrate the node:vfs virtual file system with both module loaders so files served from a mounted VFS can be resolved and loaded via require() and import, with first-class support for package.json, conditional exports, extensionless files, etc.

What lands in this PR

Toggleable loader hooks (lib/internal/modules/helpers.js). New
wrappers — loaderStat, loaderReadFile, toRealPath,
loaderLegacyMainResolve, loaderGetFormatOfExtensionlessFile,
loaderGetLayerForPath, and the four loader*PackageJSON variants —
that fall through to the existing C++ binding / fs calls when no VFS
is mounted (zero overhead on the fast path) and dispatch to VFS when
overrides are installed. Setters (setLoaderFsOverrides,
setLoaderPackageOverrides) install/clear the overrides atomically.

Consumers updated: lib/internal/modules/cjs/loader.js,
lib/internal/modules/esm/{resolve,load,get_format}.js, and
lib/internal/modules/package_json_reader.js all now go through the
wrappers. The "DO NOT depend on patchability" warnings in esm/load.js
and esm/resolve.js are preserved and now point at node:vfs and
module.registerHooks() as the formal hook mechanisms.

VFS layer ID. Each VirtualFileSystem instance is assigned a
per-process monotonically increasing layerId at construction
(vfs.layerId). The id is stable across mount/unmount cycles and
shows up in:

the NODE_DEBUG=vfs register/deregister events;
the ERR_INVALID_STATE message thrown on overlapping mounts;
the ?vfs-layer=N URL tag described below.

Scoped cache purging on unmount(). Replaces the earlier global
flush. CommonJS caches (require.cache, the CJS stat cache, the
helpers.js realpath cache, the package_json_reader caches) are
filtered with vfs.shouldHandle(filename). ESM URLs are tagged at
resolve time with ?vfs-layer=N (matching the cache-busting pattern
used by HMR tooling) and the cascaded loader's loadCache is purged
by that tag. Multi-mount setups no longer pay a cross-VFS cache-warmup
penalty on a single unmount, and ESM modules loaded from a VFS become
reachable for purge instead of leaking forever.

User-visible change: import.meta.url for VFS-loaded ES modules ends
in ?vfs-layer=N. CommonJS __filename / module.filename are
unchanged.

Docs. doc/api/vfs.md gains a Mounting section
(mount()/unmount()/mounted/mountPoint/layerId) and a Module
loader integration section covering the resolution behavior, the URL
tag, and the scoped cache purge.

Tests under test/parallel/ (all gated by --experimental-vfs):

test-vfs-require.js, test-vfs-import.mjs,
test-vfs-module-hooks.mjs — CJS / ESM / conditional-exports
resolution against a VFS.
test-vfs-package-json.js, test-vfs-package-json-cache.js,
test-vfs-invalid-package-json.js — package.json parsing,
validation parity with the native binding, cache invalidation.
test-vfs-module-hooks-cleanup.js — register/deregister cycles,
cleanup, missing-main ESM error shape.
test-vfs-layer-id.js — layerId monotonicity and stability
across mount/unmount; presence in overlap errors.
test-vfs-scoped-cache-purge.js — multi-mount isolation;
import.meta.url URL tag visibility.

Out of scope (separate follow-ups)

SEA VFS integration (the useVfs config flag, asset-as-main).
Migration of the C++ package_configs_ cache to the same scoped
purge approach.
Compile-cache / source-map-cache scoping.
Permission-model integration (--allow-fs-vfs boundary checks) is
intentionally not extended beyond the existing gate.

nodejs-github-bot · 2026-05-30T05:41:54Z

Review requested:

@nodejs/loaders

codecov · 2026-05-30T06:50:02Z

Codecov Report

❌ Patch coverage is 90.71207% with 60 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.32%. Comparing base (39b481b) to head (51b033a).
⚠️ Report is 26 commits behind head on main.

Files with missing lines	Patch %	Lines
lib/internal/vfs/setup.js	85.54%	59 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main   #63653      +/-   ##
==========================================
- Coverage   91.96%   90.32%   -1.64%     
==========================================
  Files         379      732     +353     
  Lines      166638   237271   +70633     
  Branches    25497    44727   +19230     
==========================================
+ Hits       153242   214325   +61083     
- Misses      13099    14656    +1557     
- Partials      297     8290    +7993

Files with missing lines	Coverage Δ
lib/internal/modules/cjs/loader.js	`98.15% <100.00%> (+18.48%)`	⬆️
lib/internal/modules/esm/get_format.js	`94.83% <100.00%> (+21.03%)`	⬆️
lib/internal/modules/esm/load.js	`91.47% <100.00%> (+8.22%)`	⬆️
lib/internal/modules/esm/resolve.js	`99.04% <100.00%> (+11.14%)`	⬆️
lib/internal/modules/helpers.js	`98.75% <100.00%> (+7.70%)`	⬆️
lib/internal/modules/package_json_reader.js	`99.47% <100.00%> (+11.86%)`	⬆️
lib/internal/vfs/setup.js	`86.14% <85.54%> (-0.55%)`	⬇️

... and 472 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

mcollina · 2026-05-30T10:27:16Z

@joyeecheung take a look, should be easier to review.

@jsumners-nr

Restore the "DO NOT depend on the patchability" warnings in esm/load.js and esm/resolve.js that were dropped along with the fs imports. The warning still applies; it now also points at node:vfs as one of the formal hook mechanisms callers should reach for instead. Addresses review feedback from @jsumners-nr in nodejs#63653

nodejs-github-bot · 2026-06-03T08:04:09Z

CI: https://ci.nodejs.org/job/node-test-pull-request/73901/

nodejs-github-bot · 2026-06-03T11:52:41Z

CI: https://ci.nodejs.org/job/node-test-pull-request/73902/

joyeecheung

A design question recently occurred to me: have we explored the versioning of the mounting?

joyeecheung · 2026-06-03T12:09:57Z

+  const { clearPackageJSONCache } = require('internal/modules/package_json_reader');
+  clearPackageJSONCache();
+  // The ESM cascaded loader's loadCache is intentionally NOT cleared here:
+  // clearing it mid-flight (while another import() is awaiting nested


Isn't it also the case for require() anyway? (You can unmount it at the top level of a CJS module while the graph is loading and things can be broken there, it's always up to the user to ensure they don't unmount from an unsafe place).

An alternative approach would be to add a cache-busting search params to avoid the stale caches, which solves the identity problem. This is also what the test mock module does and how some user-land HMR works, see #61767 (comment)

I wanted to avoid search params because in my experience they are unreliable (in a graph) and I wanted to minimize memory leaks. Not sure if it's totally possible.

Doesn't this now leak all the time whenever new ESM are loaded through the VFS? Unless the identifiers are reused by subsequent mounting (i.e. graph 1, 2, 3 overlaps), the modules stay stale in the cache. With a search param in the identifier I think it will be possible walk through the caches and purge everything identified by an umounted VFS version in the search params during unmounting. Not that it eliminates the leaks (#63186 will continue to be an issue), but at least it reduces the leaks when the graphs don't overlap?

mcollina · 2026-06-03T12:40:18Z

A design question recently occurred to me: have we explored the versioning of the mounting?

what do you mean? You mean multiple vfs layers on top of each other?

joyeecheung · 2026-06-03T13:02:48Z

what do you mean? You mean multiple vfs layers on top of each other?

For the stacks to have some kind of version number/ID to identify the current status?

BTW I just noticed that there's no mention of unmount() and mount() in the VFS docs..

mcollina · 2026-06-03T20:10:47Z

BTW I just noticed that there's no mention of unmount() and mount() in the VFS docs..

I did purge them when doing the splitting; I forgot to bring them back. I'll add them to this PR.

mcollina · 2026-06-03T20:11:45Z

For the stacks to have some kind of version number/ID to identify the current status?

No but we totally should.

Route loader fs and package.json operations through toggleable wrappers so the VFS can resolve and load modules from mounted paths. When no VFS is mounted, the wrappers take a null-check fast path with zero overhead. Hooks: - loaderStat / loaderReadFile / toRealPath / loaderLegacyMainResolve / loaderGetFormatOfExtensionlessFile in lib/internal/modules/helpers.js, consumed by cjs/loader.js, esm/resolve.js, esm/load.js and esm/get_format.js. - loaderReadPackageJSON / loaderGetNearestParentPackageJSON / loaderGetPackageScopeConfig / loaderGetPackageType, consumed by package_json_reader.js. - setLoaderFsOverrides / setLoaderPackageOverrides install / clear all hooks; clearRealpathCache exposes the helpers.js realpath cache so deregister can flush it. lib/internal/vfs/setup.js installs the overrides on first registerVFS and clears every JS-side loader cache (CJS _pathCache, CJS stat cache, realpath cache, package.json cache) on every deregister. The overrides themselves are uninstalled when the last VFS is removed so the fast path is fully restored. legacyMainResolve / extensionless-format behavior matches the C++ binding; package.json validation matches src/node_modules.cc (silently omit non-string main, throw on non-string name/type, etc). The "DO NOT depend on patchability" warnings in esm/load.js and esm/resolve.js are preserved and now point at node:vfs and module.registerHooks() as the formal hook mechanisms. Tests cover require / import / module-hooks / package.json / cache invalidation / cleanup-cycle scenarios under --experimental-vfs. Signed-off-by: Matteo Collina <hello@matteocollina.com>

Each VirtualFileSystem now exposes a per-process monotonically increasing `layerId`, assigned at construction. The id is stable across mount/unmount cycles for the lifetime of the instance and surfaces in: - debug() output for register / deregister so the layer stack is visible when NODE_DEBUG=vfs is enabled; - the overlap ERR_INVALID_STATE message, which now names the layer ids of the conflicting mounts. The id is the building block for tagging cache entries with the owning VFS, which a follow-up will use to replace the global loader-cache flush in deregisterVFS with a scoped purge. Refs: nodejs#63653 Signed-off-by: Matteo Collina <hello@matteocollina.com>

Replace the global loader-cache flush in deregisterVFS with a scope-purge that only drops entries owned by the unmounting VFS. Per-layer ownership is determined two ways: - For CJS-style filename-keyed caches (Module._cache, Module._pathCache, the CJS stat cache, the helpers.js realpath cache, and the package.json caches) entries are filtered with `vfs.shouldHandle(filename)`. __filename stays a clean absolute path so user code that does `path.dirname(__filename)` or similar is unaffected. - For the ESM cascaded loader's loadCache, entries are tagged at resolve time: when finalizeResolution() detects the resolved path is VFS-owned (via the new loaderGetLayerForPath hook), it appends `?vfs-layer=<id>` to the URL. The tag surfaces in `import.meta.url`, matching the cache-busting pattern used by HMR tooling. On deregister, entries whose URL carries the tag for the unmounting layer are deleted. Multi-mount setups no longer pay the cross-VFS cache-warmup penalty when a single VFS unmounts, and ESM modules loaded from a VFS become reachable for purge instead of leaking forever in the cascaded loader. New helpers exposed for VFS: - cjs/loader.js: clearStatCacheForVFS - helpers.js: purgeRealpathCacheForVFS, loaderGetLayerForPath - package_json_reader.js: purgePackageJSONCacheForVFS Adds test-vfs-scoped-cache-purge covering both the multi-mount isolation and the import.meta.url tag visibility. Refs: nodejs#63653 Signed-off-by: Matteo Collina <hello@matteocollina.com>

nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels May 30, 2026

mcollina requested review from ShogunPanda, jasnell and joyeecheung May 30, 2026 10:26

ShogunPanda approved these changes May 30, 2026

View reviewed changes

jsumners-nr reviewed Jun 1, 2026

View reviewed changes

Comment thread lib/internal/modules/esm/load.js

Comment thread lib/internal/modules/esm/resolve.js

mcollina force-pushed the vfs-module-loader-integration branch from 6321e08 to 51b033a Compare June 1, 2026 15:54

ShogunPanda approved these changes Jun 2, 2026

View reviewed changes

mcollina added the request-ci Add this label to start a Jenkins CI on a PR. label Jun 3, 2026

github-actions Bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Jun 3, 2026

joyeecheung reviewed Jun 3, 2026

View reviewed changes

mcollina added 2 commits June 3, 2026 22:15

mcollina force-pushed the vfs-module-loader-integration branch from 51b033a to 294a19c Compare June 4, 2026 07:23

Uh oh!

Conversation

mcollina commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What lands in this PR

Out of scope (separate follow-ups)

Uh oh!

nodejs-github-bot commented May 30, 2026

Uh oh!

codecov Bot commented May 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mcollina commented May 30, 2026

Uh oh!

Uh oh!

Uh oh!

nodejs-github-bot commented Jun 3, 2026

Uh oh!

nodejs-github-bot commented Jun 3, 2026

Uh oh!

joyeecheung left a comment

Choose a reason for hiding this comment

Uh oh!

joyeecheung Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

mcollina Jun 3, 2026

Choose a reason for hiding this comment

Uh oh!

joyeecheung Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mcollina commented Jun 3, 2026

Uh oh!

joyeecheung commented Jun 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mcollina commented Jun 3, 2026

Uh oh!

mcollina commented Jun 3, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

mcollina commented May 30, 2026 •

edited

Loading

codecov Bot commented May 30, 2026 •

edited

Loading

joyeecheung Jun 3, 2026 •

edited

Loading

joyeecheung commented Jun 3, 2026 •

edited

Loading